So I want to talk to you about SSD data evaporation.
I'm Sam Bowne. I teach at City College San Francisco for the moment.
And so let's talk a bit about data remnants.
This is an old issue.
If you have one of these magnetic hard drives and you write a file on the disk
and you delete the file, it just remains on the disk.
And if you reformat the disk, the file just remains on the disk.
This is computer forensic. People love this because you can get the evidence
from perps that think they've deleted stuff.
The only time that data gets erased is when you write on top of those sectors
where the data was stored.
So that's good, clean fun.
And, you know, we know these things. I'm not going to bother demonstrating them.
If you empty the recycle bin, that doesn't do anything.
Formatting the drive doesn't do anything.
Those just mark those clusters available for all practical purposes,
but they don't erase the data.
So forensics people have gotten used to a couple of luxurious things
that computer forensics people do
that other kinds of people don't do.
That other kinds of forensics people don't have.
One of them is they can recover deleted stuff.
Another thing is they can make an image of the drive
and calculate an MD5 hash,
and they can make another image of the drive,
and it's exactly the same.
And they can do that forever until the drive has a mechanical failure or something.
So computer forensics has been this beautifully clean,
mathematically precise branch of forensic science.
And that time is pretty much over.
And, of course, another fun thing you can do is you can recover deleted data,
which is more important.
It's more common for most people in computer forensics.
So if you have a PC and you want some files back,
you can use free things like Rekuva.
If you have a Mac, you can use Disk Drill.
These will bring back your deleted files, which is great.
And there's a bunch of people who make a lot of money doing this,
like Drive Savers.
It's a great company.
We had a tour of them, and they will get your stuff back
when your drive fails, which is an enormously valuable service for people.
But now we're moving to SSDs, like what I'm using right now.
This is a MacBook Air, and the Ultrabooks are SSDs in your iPhones,
in your iPads, and they're an ever-increasing part of the market.
They're the blue bar here, expected light blue bar.
Something like 40% of all storage will be on SSDs pretty soon.
I switched to them almost completely because they're wonderful.
They're fast, but they're designed to save data efficiently
without any regard for remnants.
Remnants is just an artifact of the technology of magnetic hard drives.
SSDs have other constraints.
And one of the main constraints of them is that you cannot erase
one page of an SSD.
You have to erase an entire block, which, as you can see here, is many pages.
And you can't erase it, and there's many things.
You have to erase an SSD block before you can write on it,
and you can only erase it so many times before you've
run out the SSD and break it.
So what you have to do, there's firmware, proprietary processes running inside
the firmware and the SSD, which erase a block of pages when they decide in their wisdom
that that block has had enough files deleted that what remains doesn't matter very much.
It'll move it somewhere else.
So this means that erasure has to happen before you write.
And, in fact, there's a garbage collection process running in the background
which erases things when the wisdom of the SSD controller says it's time to erase them.
It forensically wipes them.
So if you delete files on an SSD and wait, they really do vanish sometimes.
And sometimes they don't.
It gets complicated.
And so this was called self-corrosion.
Data evaporation seemed like a better name for me.
So let me do something here just to get started.
Now, if you do the simplest possible test of this, that would be to just put some stuff
on the desktop, and that's what I've done here.
So I have a folder called spam demo, which is empty, but I deleted the contents of this
folder at 4.05, which is now about half an hour ago.
And I have another folder here called spam 2, which has four files, each 200 megabytes.
By the way, when you do this, you have to have a lot of data.
Make sure you have about a gigabyte of data.
Otherwise, you won't have enough to see any significant data evaporation because it has
to be enough.
Now I'm going to put it in the recycle bin and empty the trash.
So that will ‑‑ are you sure you want to permanently erase?
We've all seen that message, and on magnetic hard drives you see that message, but it's
a lie.
But on SSDs, it is not as much of a lie.
So now let's run disk drill, which will recover deleted things off the disk.
Handy for utility.
The quick scan is good enough.
And it will take about a minute or two to run.
And we'll see what it finds.
And maybe ‑‑ yeah, I'm impatient enough to go back to the slides while this happens.
All right.
And here I'm going to just cut ahead to the chase.
What happens ‑‑ I did this many times sitting early in the morning at Starbucks
a few months ago.
The time it takes to erase the files I've deleted and really remove them is random.
Up to an hour on the Mac.
So the quick scan is finished.
Let's see what it found.
It found users, my name, desktop, spam 2.
It found all five files in spam 2.
But the ones in the old one.
The folder folder there are all gone.
There were five files.
I deleted them half an hour ago.
Now they're completely gone and unrecoverable.
That's the essence of this talk right there.
The only remaining interesting fact is how strange and random this is.
So I have all five files there.
I'll run this thing again at the end and we'll probably see that some of them are gone by
then.
Although probably not all of them.
So those are the results.
You see frequent result is it erases some of the files but not all of them.
And we'll see if it's gone by then.
and then another pass comes through later.
I'm not able to detect any pattern here.
So in the wisdom of the people that made the controller
for the Mac SSD,
it can take up to an hour for it to complete garbage collection
for things on the desktop.
Now, you can run this command
and see if your machine is supporting trim.
In order for this to happen,
something has to happen with SSDs
that does not happen with magnetic hard drives at all.
They have to know when you delete a file.
Normally, your drive does not know
when the operating system has deleted a file.
But SSDs need to know when you delete a file,
and you do that through the trim command,
which is only supported by the very latest versions
of operating systems,
and only if you have your drive running in SATA mode and AHCI.
Here's the operating system versions that you have to have.
And if you satisfy all those conditions
and you also have the very latest partition format,
then you may observe evaporation.
But you can't control the timing,
and you can't turn it off.
So here's some more examples.
You can't run it through USB,
and you can't run it through PCI Express or RAID.
But if you don't break any of those large number of rules,
then you will have the phenomenon
that deleted files are vanishing.
So this means if you were going to testify in court,
for example, evidence that you find in computer forensics,
you're going to have to be able to explain what happened here
because it's going to mess up your traditions.
Because if you make an image of an SSD
and calculate the MD5,
as soon as you put the power on
to the SSD,
even though you have a hardware write blocker,
the data on the SSD is changing.
The firmware is evaporating away that data
while you image it,
and when you make another copy,
you don't get the same MD5.
So that is going to make your evidence appear wrong,
and you're going to have to be able to explain this.
And when I took computer forensics classes,
my instructor has made it very clear to me,
this is true,
the reason you are an expert witness
is because you are allowed to have opinions,
but those opinions must be based on experience,
not hearsay.
So you cannot quote something you read in a book
or something a teacher gave you,
you have to say,
I tested it myself,
and this is how it works.
And therefore you have to have testing tools.
So I made a testing tool
to make this easier
because it's obvious to me
that people are going to have to test
the exact drives that they want to testify about
if they want to explain this stuff,
since it depends on everything.
So let me show you the tool I made
to check on the Mac
because it's kind of fun,
at least for a demo.
I wrote a little command line tool called EVAP.
And,
let me see,
I've got my window to come to the front.
This is just a bash shell script,
there's not much to it.
Let me put in a password.
All right.
So,
it has a few options here.
Now, in order to run this tool,
now what I did before
was a demonstration putting a folder on my desktop.
But for this tool,
I create a partition just for this purpose.
So I have a 500 gigabyte Apple SSD here.
And if you look at the partitions,
here's the big one,
and here's the little one.
I have a one,
one gigabyte partition I created just for testing.
And you have to do that if you want to do this one
because I'm following a 2010 paper that started this
and I found something that caught my attention.
So if I format that partition as a journaling HFS plus,
the very latest Mac format with E,
that will format that partition.
And then I can write test files on that partition with W.
And when I scan it,
I'm going to scan the entire partition
and print,
80 individual bytes evenly across it.
So you get a sort of overview of what's on there.
And what I did was write a bunch of files full of ASCII characters
so they go in the alphabetical order
so you can see what's on there.
There's a bunch of files on there filling it up in this pattern.
Now, if I delete those files with D and then scan it again,
you see what happens.
They're all gone.
Now, if I write them on there again and scan them,
and delete them and then scan them and they're all gone again,
which there's a fly in the ointment here.
I'm frequently able to show you that there's some of that left.
It didn't really get them all and it's kind of a random process.
Sometimes I can see some of those letters left and sometimes I can't.
But anyway, what's even more fun is to put it in a different format.
If you make it in an older Macintosh format,
the non-journaling file system with F,
and then write that data and then scan it,
the data is on there.
If you delete that data and then scan it,
it's all still there and it will stay there forever
just like a magnetic hard drive.
So this process is not complete and it's very hard to predict.
And by the way, if you're a crook and you want to not get caught,
you can't trust this evaporation to thoroughly remove all the data either
because some of the data is still there.
The data you put in there will not fill enough of those blocks
and it will decide to leave them and wait until later.
So it does not erase 100 percent of the data.
And I have another format, some more commands in here
that take a little longer to run where you fill the entire thing with Xs
and then erase it and then measure how many Xs are left.
And you'll find a significant number of them left.
So it's an important thing to realize.
And that's the main point here.
All right.
Now I had another demo which is not going to work.
My SSD has failed.
But I want to point out there are two cases here.
On the Mac's desktop it takes up to an hour for these things to evaporate.
On the separate partition it takes less than one second.
I can't measure the time at all.
They're instantly gone.
If you buy a Corsair SSD and put it on a PC it takes 15 seconds,
which makes an entertaining demo.
You can put it in a hex viewer and watch them
and after 15 seconds they just vanish.
So I can't give you that demo because my SSD just failed.
And I think that's all I have to tell you.
Are there any questions?
Well, if there aren't any questions in here,
I'll just hang out in the hallway to see if anybody wants to hear any more about this.
What's that?
Immigration?
I'm sorry.
I can't hear the question.
I'm sorry.
I still can't hear the question.
Why don't you come up here?
Is this true for just MLC drives?
I do not know.
No.
Your X was time.
Secure delete, you say.
Well, here, what was your question?
Your X was time.
Why not guessing time and take bets?
Guessing time and take bets.
Oh.
All right.
Oh, by the way, I said I'd run this drill again.
Let me run this drill again and see if anything interesting happened there.
But I think it hasn't been long enough.
Let's try this again and see what happens.
Anyway, you had something?
A secure erase is just writing on top of the data, right?
Yeah.
No, it doesn't.
A secure erase will not erase an SSD because SSDs have extra bytes.
If you buy a 100 gig SSD, you really get 110 or 115, and the sectors are invisibly mapped by the controller.
So when you erase them, you don't ever get the whole thing.
And there is no tool.
There's no tool that will erase the entire contents.
You can't write ‑‑ you can't access all the sectors.
Exactly.
When you write data, it's going to different sectors than you think it is.
So there is no ‑‑ the only way to securely erase an SSD is to grind it up physically or to replace the firmware with hacked firmware.
Let me just see what came here.
Now they're all gone.
There's nothing on the desktop.
You're on to it here.
This is what I've had iPhones do.
You turn on encryption before you ever save any data, and then when you want to erase it, you erase the key.
That works.
But there's no way to actually erase all the data on there because some of it is going to sectors which are then mapped to be invisible to the drive.
The same thing goes for the MacBook Air that you ‑‑
Yes.
The same thing.
This MacBook Air, I should grind it up mechanically if I try to pass it on to a student.
There's no way to clean it.
Yeah.
Yeah.
Unless I turn on encryption before you start, and that's what iPhones do.
Yeah.
When you were trying to ‑‑
Mm‑hmm.
You think maybe that's a result of there's not enough activity going on, so the garbage collection is running.
It's a good question.
It's a good question.
Why did I not see the leftover letters?
Sometimes I do and sometimes I don't, and I'm always working the same on a completely empty partition that's completely reformatted.
The results are not always the same.
And I do not know what causes it.
That's ‑‑ the main thing I discovered is you really have to try it under your conditions to know what's going to happen.
Yes.
Apple could tell you, but then there's a bunch of other SSD brands you wouldn't know about them.
Yeah.
Yeah.
My other question was, when you're doing the research in terms of whether or not the system is handling the data in the right way or not.
I don't know the answer.
He's asking if you would turn off garbage collection to save power.
I do not know if the computer can do that.
It sounds like a good idea to me.
But I haven't heard any ‑‑ I haven't read anything about being able to do that.
It sounds like a good idea.
Yeah.
Maybe we ought to gather in the hallway when we get out of the way to the next person here.
All right.
